Skip to content

Add analyzer diagnostics (AOTSG0001-AOTSG0005) to MSTest.AotReflection.SourceGeneration#9014

Draft
Evangelink wants to merge 7 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/aot-reflection-srcgen-diagnostics
Draft

Add analyzer diagnostics (AOTSG0001-AOTSG0005) to MSTest.AotReflection.SourceGeneration#9014
Evangelink wants to merge 7 commits into
microsoft:mainfrom
Evangelink:dev/amauryleve/aot-reflection-srcgen-diagnostics

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Adds analyzer-style diagnostics emitted by MSTest.AotReflection.SourceGeneration when it encounters a [TestClass] shape it cannot lower into the generated registry.

Previously these unsupported shapes were silently dropped from the generated registry, so the test author had no signal that the source generator had skipped their type or member.

New diagnostics

ID Severity Triggers when …
AOTSG0001 Warning [TestClass] is static and cannot be instantiated.
AOTSG0002 Warning [TestClass] has unbound type parameters, either directly or because an enclosing type is generic.
AOTSG0003 Warning [TestClass] is not reachable from generated code (file-local, private/private protected, or nested in same).
AOTSG0004 Warning [TestMethod] is itself generic and cannot be invoked without runtime instantiation.
AOTSG0005 Warning A [TestMethod] or constructor parameter uses ref / in / out.

When any of these fire, the offending type or member is skipped from the emitted registry instead of producing invalid C#. For type-level diagnostics (AOTSG0001AOTSG0003) only the first applicable reason is reported to avoid noise.

Implementation notes

  • New Diagnostics/ folder with DiagnosticDescriptors (5 descriptors with GetById lookup), DiagnosticInfo (value-equatable payload using EquatableArray<string>), and LocationInfo (value-equatable Location surrogate). The equatable shape keeps the incremental generator's caching working.
  • MSTestReflectionMetadataGenerator pipeline now produces a TestClassTransformResult(TestClassModel? Model, EquatableArray<DiagnosticInfo> Diagnostics); diagnostics are reported through a SelectMany branch while model emission filters out null models.
  • Accessibility check walks the full ContainingType chain and IsFileLocal flag. ProtectedOrInternal (= protected internal) is accepted; ProtectedAndInternal (= private protected) is rejected.
  • TestClassModelBuilder.Build now takes a diagnostics out-param so member-level reasons (generic methods, by-ref parameters) can be surfaced for both methods and constructors. Offending members are dropped from the model.
  • Adds AnalyzerReleases.{Shipped,Unshipped}.md referenced as AdditionalFiles, required by EnforceExtendedAnalyzerRules + Microsoft.CodeAnalysis.Analyzers (RS2008).

Tests

  • 10 new tests covering positive and negative cases for every diagnostic ID, including the multi-parameter ref/in/out case (one diagnostic per offending parameter) and the inaccessibility chain (outer-private wins over inner-public).
  • Two existing tests (Generator_SkipsStaticTestClass, Generator_SkipsGenericTestClass) updated: they previously asserted "no diagnostics" because the cases were silently filtered; they now assert the appropriate AOTSG#### diagnostic is reported.
  • 50/50 tests pass.

Dependencies

This PR is stacked on top of #9013 (async invoker). Once that lands the diff here will collapse to the analyzer/diagnostics code only.

Part of #1837.

Amaury Levé and others added 7 commits June 10, 2026 14:42
Adds a focused unit-test project for the AotReflection source generator PoC introduced in microsoft#8574. The PoC had no test coverage until now.

Coverage highlights (13 tests):

- Support types emission (TestClassReflectionInfo, TestMethodReflectionInfo, TestPropertyReflectionInfo, TestConstructorReflectionInfo).

- Registry emission shape and namespace (MSTest.SourceGenerated).

- Empty registry when no [TestClass] is present.

- Skipping of static / abstract / open-generic test classes.

- Constructor invoker, parameter types / names, async return shape.

- Class-level attribute capture; property getter & setter delegate text.

- Compile-clean snapshot (catches CS errors the generator may introduce).

- Incrementality: support-types step is cached when input is unchanged.

Also:

- Adds MSTest.AotReflection.SourceGeneration to TestFx.slnx and MSTest.slnf (missing since microsoft#8574).

- Adds [InternalsVisibleTo] for the new test project (generator class is internal sealed).

Part of microsoft#1837.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…8639)

The PoC's TestClassModelBuilder built its fully-qualified type names with SymbolDisplayMiscellaneousOptions.IncludeNullableReferenceTypeModifier, then fed the resulting FQN into both casts (where '?' is harmless and merely cosmetic, since the emitted setter already uses 'value!') and 'typeof(...)' expressions (where '?' on a reference type is invalid C# and produces CS8639).

Removing the flag fixes 'typeof(string?)' / 'typeof(MyRef?)' while preserving 'typeof(int?)' (nullable value types are rendered via UseSpecialTypes, which is unaffected).

Adds a focused regression test that runs the Roslyn compiler over the generated source and asserts both the textual shape ('typeof(global::Sample.TestContext)', 'typeof(string)', 'typeof(int?)') and the absence of any compile errors.

Discovered while building tests for microsoft#8574; depends on microsoft#9004 for the test infrastructure.

Part of microsoft#1837.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Today TestClassModelBuilder only enumerates members directly declared on the
`[TestClass]` type. As soon as a fixture extends a base class, MSTest members
(`[TestInitialize]`, `[TestCleanup]`, `[TestMethod]`, the `[TestContext]`
property, ...) declared on the base disappear from the generated registry.

This change makes the builder walk the inheritance chain (stopping at
`System.Object`):

* Methods and properties are folded from base types into the model.
* Iteration is derived-first; an override or `new`-shadowed member with the
  same signature/name wins over the base declaration. The signature key
  includes ref-kinds so genuine overloads survive.
* Attributes are collected across the `OverriddenMethod` / `OverriddenProperty`
  chain (deduped by attribute class FQN) so an `override` that does not
  re-apply `[TestMethod]` still sees the base attribute - matching the
  runtime `GetCustomAttributes(inherit: true)` semantics.
* Accessibility is broadened to include `Protected` / `ProtectedOrInternal` /
  `ProtectedAndInternal` so abstract bases can expose their hooks to the
  emitted code (which lives in the consumer's assembly).
* Constructors are NEVER inherited (only taken from the leaf type).

Adds 9 new tests covering: inherited methods, multi-level inheritance,
overridden virtual (attribute inheritance + de-dup), `new`-hidden methods,
overload preservation, inherited properties, no inherited constructors,
abstract-base fold-down, and not walking past `System.Object`.

Part of microsoft#1837. Depends on microsoft#9004 (test project), microsoft#9005 (typeof nullable).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds an AssemblyAttributes property to the emitted MSTestReflectionMetadata registry containing all attributes declared with [assembly: ...] in the same compilation. The attribute payload is built via the existing AttributeApplicationModel pipeline (reused from class/method attribute emission), so adapters can iterate without calling Assembly.GetCustomAttributes at runtime.

Part of microsoft#1837. Stacked on microsoft#9004, microsoft#9005, microsoft#9006.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft#1837.

Adds compile-time materialization of `[DataRow]` attribute applications on
`[TestMethod]` members. The generator now emits a `DataRows` property on
each `TestMethodReflectionInfo` containing a flat `IReadOnlyList<object?[]>`
mirroring the runtime shape of `DataRowAttribute.Data`, so a consumer can
iterate parameterised cases without re-reading the attributes via reflection.

Highlights:

- New `DataRowModel(EquatableArray<TypedConstantModel> Arguments)` capturing
  one row of arguments per attribute application.
- `BuildDataRows` detects `Microsoft.VisualStudio.TestTools.UnitTesting.DataRowAttribute`
  applications and flattens the variadic `params object?[] moreData` tail
  back into the row so the emitted array matches `DataRowAttribute.Data`
  rather than preserving a nested array.
- Inheritance-aware: reuses the inherited attribute walk introduced in microsoft#9006,
  so `[DataRow]` applied on a base method (when the override is virtual) is
  still picked up.
- Emitter always emits `DataRows` (empty array for non-data-driven tests)
  for shape parity with the other `TestMethodReflectionInfo` properties.

Deferred to a follow-up: `[DynamicData]` materialization. Resolving the
data source method/property/field at compile time is materially more complex
(handles `Method`/`Property`/`Field`/`AutoDetect` source kinds plus
`object[]` / `IEnumerable<object[]>` return shapes) and warrants its own
PR.

Tests:

- `Generator_EmitsEmptyDataRows_WhenMethodHasNoDataRow`
- `Generator_CapturesSingleDataRow_WithScalarArgs`
- `Generator_CapturesMultipleDataRows_InDeclarationOrder`
- `Generator_FlattensParamsArrayInDataRow`
- `Generator_HandlesNullValueInDataRow`

Total: 32/32 passing.

Depends on microsoft#9004, microsoft#9005, microsoft#9006, microsoft#9007.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Part of microsoft#1837.

Changes the emitted `TestMethodReflectionInfo.Invoke` delegate from
`Func<object?, object?[]?, object?>` to `Func<object?, object?[]?, Task>`
so the caller can `await` a single Task regardless of the underlying test
method's signature.

Per return shape:

- `void` / non-Task sync: `{ call; return Task.CompletedTask; }` — no
  allocation, no extra wrapping.
- `Task` / `Task<T>`: `{ Task? __t = call; return __t ?? Task.CompletedTask; }`
  — forward the Task; tolerate a misbehaving test returning `null` rather
  than NRE.
- `ValueTask` / `ValueTask<T>`:
  `{ var __vt = call; return __vt.IsCompletedSuccessfully ? Task.CompletedTask : __vt.AsTask(); }`
  — fast path for synchronous completion skips `AsTask()` allocation.
- Non-void non-Task sync (e.g. `int Test()`):
  `{ _ = call; return Task.CompletedTask; }` — value discarded, side
  effects retained.

Support type updated to declare `Invoke` as `Func<object?, object?[]?, Task>`
with default `static (_, _) => Task.CompletedTask`. Both the support-types
file and the registry file now import `System.Threading.Tasks`.

Also drops a duplicate blank line left over from PR-A4 (SA1507).

Tests:

- `Generator_SupportType_DeclaresInvokeAsTaskReturning`
- `Generator_InvokerForVoidMethod_ReturnsCompletedTask`
- `Generator_InvokerForTaskMethod_ForwardsTask`
- `Generator_InvokerForTaskOfTMethod_ForwardsTask`
- `Generator_InvokerForValueTaskMethod_UnwrapsViaAsTask`
- `Generator_InvokerForValueTaskOfTMethod_UnwrapsViaAsTask`
- `Generator_InvokerForNonVoidSyncMethod_DiscardsResultAndReturnsCompletedTask`
- `Generator_EmittedRegistry_ImportsSystemThreadingTasks`

Total: 40/40 passing (1 existing test updated for new shape).

Depends on microsoft#9004, microsoft#9005, microsoft#9006, microsoft#9007, microsoft#9011.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lass shapes

Adds analyzer-style diagnostics emitted by MSTest.AotReflection.SourceGeneration when it encounters a [TestClass] shape it cannot lower into the generated registry:

- AOTSG0001 - static [TestClass] cannot be instantiated.
- AOTSG0002 - open-generic [TestClass] (directly or via outer generic).
- AOTSG0003 - inaccessible [TestClass] (file-local, private/protected nested, or nested in an inaccessible outer).
- AOTSG0004 - generic [TestMethod] cannot be invoked without runtime instantiation.
- AOTSG0005 - [TestMethod]/constructor parameter uses 
ef/in/out.

When any of these fire, the offending type or member is skipped from the emitted registry instead of producing invalid C#. Adds 10 new tests covering positive and negative cases, plus AnalyzerReleases.{Shipped,Unshipped}.md to satisfy RS2008.

Part of microsoft#1837.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 10, 2026 15:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances MSTest.AotReflection.SourceGeneration by emitting analyzer-style diagnostics (AOTSG0001–AOTSG0005) whenever the generator encounters [TestClass]/[TestMethod] shapes it cannot safely lower into source-generated registry code, instead of silently skipping unsupported items. It also extends the generated metadata model/emitter (e.g., DataRow materialization and assembly-level attributes) and adds a dedicated unit test project to validate the resulting behavior.

Changes:

  • Add diagnostic infrastructure (DiagnosticDescriptors, equatable DiagnosticInfo + LocationInfo) and wire diagnostics reporting into the incremental generator pipeline.
  • Extend model + emitter to carry additional metadata (e.g., DataRow rows and assembly-level attributes) and adjust registry/support-types output accordingly.
  • Add MSTest.AotReflection.SourceGeneration.UnitTests with extensive generator behavior tests and include the new projects in solution files/filters.
Show a summary per file
File Description
TestFx.slnx Adds the generator and its unit tests to the main solution.
MSTest.slnf Adds the generator project to the MSTest solution filter.
src/Analyzers/MSTest.AotReflection.SourceGeneration/MSTest.AotReflection.SourceGeneration.csproj Registers analyzer release tracking files as AdditionalFiles and exposes internals to the new unit test assembly.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Model/TestClassModel.cs Extends the metadata model with DataRowModel and AssemblyMetadataModel.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Generators/TestClassModelBuilder.cs Adds member filtering + member-level diagnostics collection; builds method/property metadata across inheritance; materializes DataRow payloads.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Generators/MSTestReflectionMetadataGenerator.cs Changes pipeline to produce (Model, Diagnostics) results, reports diagnostics, and emits assembly-level attributes.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Generators/MetadataRegistryEmitter.cs Updates support types and registry emission (Task-returning invokers, DataRows, and assembly-attribute emission).
src/Analyzers/MSTest.AotReflection.SourceGeneration/Diagnostics/LocationInfo.cs Introduces an equatable Location surrogate for incremental caching.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Diagnostics/DiagnosticInfo.cs Introduces equatable diagnostic payloads and conversion to Roslyn Diagnostic.
src/Analyzers/MSTest.AotReflection.SourceGeneration/Diagnostics/DiagnosticDescriptors.cs Defines AOTSG0001–AOTSG0005 descriptors + lookup by id.
src/Analyzers/MSTest.AotReflection.SourceGeneration/AnalyzerReleases.Unshipped.md Adds release-tracking entries for the five new diagnostic IDs (RS2008).
src/Analyzers/MSTest.AotReflection.SourceGeneration/AnalyzerReleases.Shipped.md Adds the shipped release tracking placeholder file.
test/UnitTests/MSTest.AotReflection.SourceGeneration.UnitTests/Program.cs New MTP-based unit test runner entrypoint for the test project.
test/UnitTests/MSTest.AotReflection.SourceGeneration.UnitTests/MSTestReflectionMetadataGeneratorTests.cs Adds extensive behavior tests covering diagnostics and emitted source shapes.
test/UnitTests/MSTest.AotReflection.SourceGeneration.UnitTests/MSTest.AotReflection.SourceGeneration.UnitTests.csproj New unit test project wiring references/packages to test the generator.

Copilot's findings

  • Files reviewed: 15/15 changed files
  • Comments generated: 3

Comment on lines +147 to +153
private static bool IsAccessibleFromConsumer(ISymbol symbol)
=> symbol.DeclaredAccessibility is
Accessibility.Public
or Accessibility.Internal
or Accessibility.Protected
or Accessibility.ProtectedOrInternal
or Accessibility.ProtectedAndInternal;
Comment on lines +279 to +283
// Mirror the runtime behavior of MemberInfo.GetCustomAttributes(inherit: true): walk the
// overridden-method chain and union attributes, keeping the most-derived application when
// the same attribute type appears on multiple levels.
private static ImmutableArray<AttributeData> CollectInheritedAttributes(IMethodSymbol method)
{
Comment on lines +41 to +45
messageFormat: "[TestClass] type '{0}' is not reachable from generated code in the same assembly (file-local, or nested in a private/private-protected outer type)",
category: Category,
defaultSeverity: DiagnosticSeverity.Warning,
isEnabledByDefault: true,
description: "Generated registry code lives in the same assembly but in a different file/type and therefore cannot reference file-local types or types nested in a private (or private-protected) outer type. Make the test class — and every enclosing type — at least internal.");
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants